Contemporary supercomputer technology is often employed for the visualization of large data sets and for processing of real-time, high resolution images. This requires large image data storage and control capability coupled with the use of high resolution monitors, and high resolution motion color images that are sampled in real-time.
Many present-day supercomputers do not include a display controller. A workstation which controls a user interface with a supercomputer typically includes a graphics controller, but can display only those images generated within the workstation.
There is thus a need for a display controller separate from a supercomputer and the controlling workstation for visualizing and combining supercomputer output data, and/or high definition television (HDTV) input, on a very high resolution screen under a workstation user's control.
Requirements for such a display controller include an ability to process a variety of image or graphics visuals, an ability to accommodate a variety of screen resolutions, television standards, image sizes, and an ability to provide color control and correction. By example, the display controller should accommodate full motion video real-time animated images, still images, text and/or graphics. These images may be represented in different formats, such as RGB, YUV, HVC, and color indexed images. Different display resolutions may also need to be accommodated, such as 1280.times.1024 pixels for graphics image and 1920.times.1035 lines for HDTV. Finally, there may be a requirement to show a stereoscopic image, which consists of left and right views, that is shown at twice the speed of a normal non-stereo, or planar, image.
One problem arises when a monitor is required to display image data from a variety of sources, wherein the monitor may have a resolution different from any of the image data sources. Further complicating the display is a requirement that diverse images be video refreshed synchronously, and have a common final representation, such as RGB.
Another problem is that visuals originate from different sources, such as a television camera, a very high speed supercomputer interface, and a slower interface with the workstation host processor. It is clear that the interfaces of the multimedia display to these sources, and their data structures, are specific, but they must also coexist. For example, providing maximum throughput for a supercomputer data path must not interfere with a television data stream, in that television images cannot be delayed without losing information.
A further problem is that to overlay a plurality of diverse images is a complicated process. Simple pixel multiplexing becomes complicated in a multitasking environment, where different images and their combinations must be treated differently in different application windows.
One possible solution to these diverse problems is derived from an approach used by a variety of known multimedia display controllers. This solution treats each image source separately and stores the data of each source in a separate frame buffer. Each frame buffer may have different dimensions, that is, resolution and number of bits per pixel. All of the frame buffers are then refreshed synchronously. As can be realized, such a system is expensive and requires a complicated, high performance video data path, where all possible image combinations must be handled. Although this conventional approach may be referred to as "modular", it lacks the integration required for a truly equal functional treatment of all images from the user's point of view. Furthermore, the amount of memory required to realize the different frame buffers may be much larger than actually needed to store the images. That is, due to fixed memory chip organizations and capacities, and the diversity of image representations and formats, an inefficient use of memory may result, requiring more memory chips or modules than that actually required to store a given image.
In commonly assigned U.S. Pat. No. 4,994,912, issued Feb. 19, 1991, entitled "Audio Video Interactive Display" to Lumelsky et al. there is described method and apparatus for synchronizing two independent rasters such that a standard TV video and a high resolution computer generated graphics video may each be displayed on a high resolution graphics monitor. This is achieved through the use of dual frame buffers, specifically a TV frame buffer and a high resolution frame buffer. A switching mechanism selects which of the TV video and the high resolution graphics video is to be displayed at a given time. The graphics data is combined with the TV video for windowing purposes.
In commonly assigned U.S. Pat. No. 4,823,286, issued Apr. 18, 1989, entitled "Pixel Data Path For High Performance Raster Displays with All-Point-Addressable Frame Buffers" to Lumelsky et al. there is described a multichannel data path architecture which assists a host processor in communicating with a frame buffer. FIGS. 12, 13, and 14 illustrate a plane mode, a slice mode, and a pixel mode format which are related to the organization of the addressing of the frame buffer.
In commonly assigned U.S. Pat. No. 4,684,936, issued Aug. 4, 1987, entitled "Displays Having Different Resolutions For Alphanumeric and Graphics Data" to Brown et al. there is described a display terminal that presents alphanumeric and graphic data at different resolutions simultaneously. The durations of the individual alphanumeric and graphic dots have a fixed but non-integral ratio to each other, and are mixed together asynchronously to form a combined video signal to a CRT.
In U.S. Pat. No. 4,947,257, issued Aug. 7, 1990, entitled "Raster Assembly Processor" to Fernandez et al. there is described a raster assembly processor that receives a plurality of full motion video and still image input signals and assembles these signals into a full bandwidth color component, high resolution video output signal in standard HDTV format (i.e. NHK-SMPTE 1125-line HDTV format). A multi-media application is organized into a plurality of overlapping windows, where each window may comprise a video or a still image. A single multiported memory system is utilized to assemble the multi-media displays. Raster data is read out of the memory through a multiplexer that combines the signals present on a plurality of memory output channels into an interlaced 30 frame/second HDTV signal. A key based memory access system is used to determine which pixels are written into the memory at particular memory locations. Video and still image signal pixels require four bytes, specifically, Red (R), Green (G), and Blue (B) color component values and a key byte, the key byte containing a Z (depth) value. This patent does not address the storage of a high definition video signal or the storing and display of two real time images. Also, the provision of a multi-resolution display output is not addressed. Furthermore, the key data byte is employed for enabling memory write operations and, as a result, after the video is stored, the image within the window is fixed.
In U.S. Pat. No. 4,761,642, issued Aug. 2, 1988, entitled "System For Providing Data Communication Between A Computer Terminal And A Plurality of Concurrent Processes Running on a Multiple Process Computer" to Huntzinger there is described a system that allows a single computer to simultaneously run several processes and show the output of each process in a correspondent display screen window selected from a plurality of windows. Software includes a screen process for maintaining a subrectangular list comprising a set of instructions for allocating window portions of the screen to the displays defined by separate display lists.
In U.S. Pat. No. 4,953,025, issued Aug. 28, 1990, entitled "Apparatus For Defining an Effective Picture Area of a High Definition Video Signal When Displayed on a Screen With A Different Aspect Ratio" to Saitoh et al. there is described an apparatus for changing a video input aspect ratio. Specifically, a HDTV video signal is digitized, stored within a memory, and displayed on the picture screen of an NTSC or other conventional television monitor receiver having an aspect ratio that differs from that of the HDTV format.
In U.S. Pat. No. 4,631,588, issued Dec. 23, 1986, entitled "Apparatus and Its Method For The Simultaneous Presentation of Computer Generated Graphics And Television Video Signals" to Barnes et al. there is described a method for generating a graphic overlay on a standard video signal. The resulting video has the same resolution and timing as the incoming video signal.
In U.S. Pat. No. 3,904,817, issued Sep. 9, 1975, entitled "Serial-Scan Converter" to Hoffman et al. there is described a scan-converter display for operating with a variety of radar sweep signals or a variety of television raster sweep signals. A serial main memory is used for refreshing the display at a rate much higher than a radar data acquisition rate. A sweep format of a common display is altered so as to accommodate video from a variety of sources of different video formats.
What is not taught by these patents, and what is thus one object of the invention to provide, is a multimedia display for storing and displaying a plurality of real time images, and which furthermore enables the use of a plurality of programmable output video resolutions.
It is another object of the invention to provide a novel frame buffer organization so as to achieve an efficient use of memory devices.
It is a further object of the invention to provide for the display of image data from a plurality of image sources, including a plurality of real time image sources, with a single frame buffer.
It is another object of the invention to provide a video image storage format wherein a pixel includes R, G, B data and associated key data, the key data being used for controlling an output video data path and enabling the display of stored video images to be altered.